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What  GAO  Found 

DOD’s  process  for  justifying  multiyear  programs  leaves  questions  about  the 
appropriateness  of  some  approved  MYPs  and  the  cost  effectiveness  of 
investments  made  for  the  risks  assumed,  as  indicated  by  recent  submissions 
for  the  F-22A  and  V-22.  Although  the  law  has  clear  requirements  for  stable, 
low  risk  programs  with  realistic  cost  and  savings  estimates,  lack  of  guidance 
and  a  rigorous  process  is  not  achieving  this. 

It  is  difficult  to  precisely  determine  the  impact  of  multiyear  contracting  on 
procurement  costs.  GAO  studies  of  three  recent  MYPs  identified  unit  cost 
growth  ranging  from  10  to  30  percent  compared  to  original  estimates,  due  to 
changes  in  labor  and  material  costs,  requirements  and  funding,  and  other 
factors.  In  some  cases,  actual  MYP  costs  were  higher  than  estimates  for 
annual  contracts.  Although  annual  contracts  also  have  unit  cost  growth,  it  is 
arguably  more  problematic  for  MYP’s  because  of  the  up-front  investments  and 
the  government’s  exposure  to  risk  over  multiple  years. 

MYP  savings  were  on  average  higher  before  changes  in  law  called  for 
“substantial  savings”  rather  than  a  specific  quantitative  standard.  Other 
factors — lower  quantities  of  modern  systems  procured,  stricter  cancellation 
liability  allowances,  and  contraction  in  the  defense  industrial  base — may  have 
also  impacted  savings  by  lessening  opportunities  for  more  efficient  purchases, 
a  key  attribute  of  MYPs. 

DOD  does  not  track  multiyear  results  against  original  expectations  and  makes 
little  effort  to  validate  if  actual  savings  were  achieved.  GAO’s  case  studies 
indicate  that  evaluating  actual  MYP  results  provides  valuable  information  on 
the  veracity  of  original  estimates  in  the  justification  packages,  the  impacts  on 
costs  and  risks  from  internal  and  external  events,  and  lessons  learned  that  can 
be  used  to  improve  future  multiyear  candidates  and  savings  opportunities. 
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Why  GAO  Did  This  Study 

DOD  spends  $10  billion  annually  on 
multiyear  procurement  (MYP) 
contracts  for  weapons  systems. 

MYPs  may  save  money  through  more 
efficient  relationships  with  suppliers 
and  producers,  but  may  also  suffer 
losses  if  cancelled  and  can  limit  future 
budget  flexibility-  Recently,  Congress 
has  been  concerned  about  DOD’s 
management  of  the  process  and 
savings  realized  by  MYPs.  GAO  was 
asked  to  evaluate  DOD’s  review 
process  for  MYP  candidates;  examine 
MYP  program  outcomes;  identify  the 
impact  of  changes  to  MYP  savings 
threshold  guidance,  and  determine 
how  much  DOD  validates  MYP 
performance.  To  do  this,  GAO 
reviewed  statutes  and  other  guidance, 
held  discussions  with  relevant 
officials,  examined  DOD  budget 
justifications  and  contracts,  and 
conducted  limited  case  studies. 


What  GAO  Recommends 


GAO  is  making  recommendations 
that  DOD  improve  the  outcomes  of 
multiyear  justification  reviews  by 
improving  guidance,  providing  third 
party  validation  of  MYP  estimates, 
implementing  a  centralized 
database  for  MYP  information,  and 
conducting  after-action 
assessments  of  completed  MYP 
contracts.  DOD  concurred  with 
two  of  the  recommendations  and 
partially  concurred  with  the  other 
two,  stating  that  it  will  consider 
whether  the  delays  and  expense  of 
third  party  validation  are  warranted 
by  the  benefits  and  that  several 
factors  limit  the  usefulness  of  after¬ 
action  assessments.  GAO  believes 
its  recommendations  remain  valid 
and  can  improve  MYP  outcomes. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  GAO-08-298. 

For  more  information,  contact  Michael 
Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov. 
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Abbreviations 


DFARS 

Defense  Federal  Acquisition  Regulation  Supplement 

DOD 

Department  of  Defense 

EOQ 

economical  order  quantities 

FAR 

Federal  Acquisition  Regulation 
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Financial  Management  Regulation 

FPDS-  NG 
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Individual  Contract  Action  Report 
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Under  Secretary  of  Defense 
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Accountability  *  Integrity  *  Reliability 


United  States  Government  Accountability  Office 
Washington,  DC  20548 


February  7,  2008 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

For  the  past  25  years,  the  Department  of  Defense  (DOD)  has  used 
multiyear  contracts  to  procure  thousands  of  major  weapon  systems, 
investing  about  $10  billion  annually  in  recent  years.  Multiyear 
procurement  is  a  special  authority  to  contract  for  up  to  5  program  years  of 
requirements  in  one  year.  When  used  appropriately,  multiyear  contracting 
can  save  money  compared  to  a  series  of  annual  contracts  by  allowing 
more  economic  procurement  from  suppliers  and  more  efficient 
production.  Multiyear  contracting  can  also  entail  some  risks  of  substantial 
losses  if  a  program  is  reduced  or  a  contract  is  cancelled  early  and  can  limit 
DOD’s  and  the  Congress’  future  budget  flexibility.  To  identify  multiyear 
candidates  that  demonstrate  sufficient  benefits  and  manageable  risks,  the 
law  requires  certain  criteria  be  met  before  an  agency  can  enter  into  a 
multiyear  contract. 

During  discussions  on  the  President’s  Budget  for  fiscal  year  2007, 
committee  members  were  concerned  about  the  amount  of  projected 
savings  realized  on  multiyear  contracts  and  had  questions  about  the  DOD’s 
management  and  oversight  of  programs  with  multiyear  contracts. 
Accordingly,  you  asked  us  to  (1)  evaluate  DOD’s  review  process  for 
multiyear  procurement  candidates  submitted  to  Congress  for  approval;  (2) 
examine  cost  and  program  outcomes  on  selected  multiyear  programs;  (3) 
identify  impacts  from  changes  in  guidance  and  interpretation  of  the 
savings  requirement;  and  (4)  determine  the  extent  to  which  DOD  tracks 
and  validates  multiyear  performance. 

In  conducting  our  work,  we  identified  statutory  criteria  and  implementing 
policies  and  procedures  used  by  DOD  to  prepare  and  evaluate  multiyear 
justification  proposal  packages.  We  discussed  management  oversight, 
practices,  and  results  of  the  justification  process  with  cognizant  officials 
from  each  of  the  military  departments  and  the  Office  of  the  Secretary  of 
Defense  (OSD).  We  examined  DOD  historical  budgetary  and  contractual 
records  to  compile  and  summarize  data  on  prior  and  current  multiyear 
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programs.  We  looked  in  more  detail  at  nine  major  programs  representing 
each  military  department:  three  had  completed  multiyear  contracts,  two 
were  beginning  multiyear  contracts,  and  four  programs  were  just  recently 
authorized  by  the  Congress.  Several  programs  have  been  approved  for 
more  than  one  multiyear  contract  and,  collectively,  these  programs 
comprised  almost  one-third  of  approvals  granted  since  1996.  We 
researched  the  legislative  history  on  the  required  savings  criterion  and 
DOD’s  efforts  to  interpret  and  apply  it.  We  reviewed  and  discussed  two 
recent  multiyear  studies  conducted  by  the  defense  research  organizations, 
RAND  National  Defense  Research  Institute  and  the  Institute  for  Defense 
Analyses  (IDA).  Appendix  I  further  explains  the  report’s  scope  and 
methodology.  We  conducted  this  performance  audit  from  June  2007  to 
February  2008  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


Results  in  Brief 


Multiyear  contracting  is  big  business  and  promises  savings  at  some  risk  to 
the  government,  yet  DOD’s  management  direction  and  process  for 
justifying  multiyear  programs  to  the  Congress  is  limited,  raising  questions 
about  the  appropriateness  of  some  approved  multiyear  programs  and  the 
cost  effectiveness  of  investments  made  for  the  risks  assumed.  We 
identified  concerns  about  the  relative  stability  and  savings  potential  of  two 
recently  approved  programs,  the  F-22A  Raptor  and  V-22  Osprey.  We  found 
differences  in  how  officials  interpreted  and  applied  the  statutory  criteria 
and  in  the  methods  and  data  used  to  compute  contract  cost  and  savings. 
Further,  few  records  are  kept  to  document  decisions  and  supporting 
evidence.  The  statutory  criteria  establish  requirements  for  stable,  low  risk 
programs  with  realistic  cost  and  savings  estimates,  but  DOD  has  not 
provided  sufficient  guidance  and  a  rigorous,  disciplined  process  to  ensure 
high  quality,  consistent  decisions  supported  by  strong  empirical  evidence. 

Although  it  is  difficult  to  precisely  determine  the  impact  of  multiyear 
contracting  on  actual  procurement  costs,  our  case  studies  of  completed 
multiyear  contracts  for  the  C-17A  Globemaster,  F/A-18E/F  Super  Hornet, 
and  Apache  Longbow  Helicopter  identified  significant  unit  cost  growth, 
ranging  from  10  to  30  percent  compared  to  the  original  estimates  provided 
to  Congress.  All  three  programs — presumably  approved  based  on  their 
stability — were  significantly  impacted  during  contract  execution  by  labor 
and  material  cost  increases,  changes  in  requirements  and  funding,  and 
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other  factors  that  helped  drive  up  total  contract  costs  much  beyond 
original  projections.  Savings  also  do  not  appear  to  have  materialized  as 
expected  in  the  budget  justifications  submitted  to  the  Congress  and 
ultimately  more  funding  was  needed  to  buy  the  systems.  In  two  of  the 
three  cases,  actual  costs  for  multiyear  procurement  exceeded  original 
estimates  for  annual  contracts.  While  both  annual  and  multiyear 
contracting  are  prone  to  the  underestimation  of  costs  and  overstatement 
of  benefits  as  we  have  noted  in  our  prior  body  of  work  on  defense 
acquisitions,  the  stakes  are  arguably  higher  for  multiyear  programs 
because  of  the  increased  up-front  investment  required,  considerable  cost 
increases  if  a  program  is  significantly  restructured,  and  the  greater 
liabilities  incurred  if  multiyear  programs  are  cancelled. 

The  meaning  and  application  of  the  savings  requirement  for  multiyear 
contracts  have  evolved  over  time.  Expectations  of  10  percent  savings  or 
more  were  emphasized  during  the  1980s,  but  replaced  with  a  “substantial 
savings”  requirement  in  fiscal  year  1991.  While  a  direct  cause  and  effect 
relationship  cannot  be  demonstrated,  our  analysis  of  multiyear  programs 
approved  by  Congress  shows  that  estimated  savings  were  on  average 
higher  in  the  years  before  the  “substantial  savings”  requirement  was 
established  than  after,  although  there  were  wide  ranges  below  and  above 
10  percent  during  both  periods.  Other  factors — lower  quantities  of  modern 
systems  being  procured,  stricter  cancellation  liability  allowances,  and 
contraction  in  the  defense  industrial  base — may  also  decrease  estimated 
savings  for  current  and  future  systems  by  lessening  the  opportunities  for 
achieving  benefits  from  economic  quantity  buys,  a  key  attribute  of 
multiyear  contracts. 

DOD  does  not  have  a  formal  mechanism  for  tracking  multiyear  results 
against  original  expectations  and  makes  few  efforts  to  validate  whether 
actual  savings  were  achieved  by  multiyear  procurement.  It  does  not 
maintain  comprehensive  central  records  and  historical  information  that 
could  be  used  to  enhance  oversight  and  knowledge  about  multiyear 
performance  to  inform  and  improve  future  multiyear  procurement  (MYP) 
candidates.  DOD  and  defense  research  centers  officials  said  it  is  difficult 
to  assess  results  because  of  the  lack  of  historical  information  on  multiyear 
contracts,  comparable  annual  costs,  and  the  dynamic  acquisition 
environment.  Despite  these  limitations,  our  case  studies  indicate  that 
evaluating  the  actual  results  from  multiyear  contracting  provide  valuable 
information  regarding  the  veracity  of  original  estimates  in  the  justification 
packages,  the  impacts  on  costs  and  risks  from  internal  and  external 
events,  and  lessons  learned  that  can  be  used  to  improve  future  MYP 
candidates  and  savings  opportunities. 
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GAO  is  making  four  recommendations  to  enhance  the  multiyear 
procurement  approval  process  for  major  DOD  weapon  systems  that 
include  improving  guidance  related  to  the  multiyear  procurement  decision 
criteria,  establishing  a  third  party  validation  process  for  multiyear 
candidate  programs,  maintaining  a  central  database  for  monitoring  major 
DOD  weapon  system  multiyear  procurements,  and  conducting  after-action 
assessments  of  completed  multiyear  contracts  used  to  procure  major  DOD 
weapon  systems.  In  written  comments  on  our  draft  report,  DOD 
concurred  with  our  two  recommendations  on  improving  guidance  and 
maintaining  a  central  database.  DOD  partially  concurred  with  our  two 
recommendations  for  third  party  validations  and  after-action  assessments, 
stating  that  they  may  be  of  value  for  selected,  but  not  all,  programs.  We 
believe  that  third  party  validations  can  improve  the  consistency  and 
quality  of  cost  and  savings  estimates  that  are  integral  to  congressional  and 
DOD  decisions  on  multiyear  proposals.  Our  review  identified  inconsistent 
practices  and  varying  degrees  of  quality  and  completeness  in  the 
preparation  and  internal  review  of  initial  cost  and  savings  estimates  made 
by  the  weapon  system  program  offices.  After-action  assessments  can 
provide  a  continuing  database  of  lessons  learned  that  can  benefit  future 
programs. 


Background 


Multiyear  contracting  is  a  special  authority  for  acquiring  more  than  one 
year’s  requirements  — including  weapon  systems —  under  a  single  contract 
award  without  having  to  exercise  an  option  for  each  program  year  after 
the  first.  It  is  an  exception  to  the  full-funding  policy  that  requires  the  entire 
procurement  cost  of  a  weapon  or  piece  of  equipment  to  be  funded  in  the 
year  in  which  the  item  is  procured.  Under  a  multiyear  procurement,  DOD 
can  contract  for  up  to  5  years  of  quantities,  although  funding  is  still 
appropriated  on  an  annual  basis. 

Multiyear  procurement  can  potentially  save  money  and  improve  the 
defense  industrial  base  by  permitting  the  more  efficient  use  of  a 
contractor’s  resources.  Multiyear  contracts  are  expected  to  achieve  lower 
unit  costs  compared  to  annual  contracts  through  one  or  more  of  the 
following  sources:  (1)  purchase  of  parts  and  materials  in  economic  order 
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quantities1  (EOQ),  (2)  improved  production  processes  and  efficiencies,  (3) 
better  utilized  industrial  facilities,  (4)  limited  engineering  changes  due  to 
design  stability  during  the  multiyear  period,  and  (5)  cost  avoidance  by 
reducing  the  burden  of  placing  and  administering  annual  contracts. 
Multiyear  procurement  also  offers  opportunities  to  enhance  the  industrial 
base  by  providing  defense  contractors  a  longer  and  more  stable  time 
horizon  for  planning  and  investing  in  production  and  by  attracting 
subcontractors,  vendors,  and  suppliers.  However,  multiyear  procurement 
also  entails  certain  risks  that  must  be  balanced  against  potential  benefits, 
such  as  the  increased  costs  to  the  government  should  the  multiyear 
contract  be  changed  or  canceled  and  decreased  annual  budget  flexibility 
for  the  program  and  across  DOD’s  portfolio  of  weapon  systems. 
Additionally,  multiyear  contracts  often  require  greater  budgetary  authority 
in  the  earlier  years  of  the  procurement  to  economically  buy  parts  and 
materials  for  multiple  years  of  production  than  under  a  series  of  annual 
buys. 

Although  DOD  had  been  entering  into  multiyear  contracts  on  a  limited 
basis  prior  to  the  1980s,  the  Department  of  Defense  Authorization  Act, 
1982, 2  codified  the  authority  for  DOD  to  procure  on  a  multiyear  basis 
major  weapon  systems  that  meet  certain  criteria.  Since  that  time,  DOD  has 
annually  submitted  various  weapon  systems  as  multiyear  procurement 
candidates  for  congressional  authorization.  Over  the  past  25  years, 
Congress  has  authorized  the  use  of  multiyear  procurement  for 
approximately  140  acquisition  programs,  including  some  systems 
approved  more  than  once.  Section  2306b  of  title  10,  United  States  Code, 
governs  the  use  of  multiyear  contracting  authority  for  the  procurement  of 
property  by  DOD.  It  specifies  six  statutory  requirements,  or  criteria,  that 
an  acquisition  program  is  expected  to  meet  in  order  to  be  considered  for 
multiyear  contracting.  These  criteria  are  listed  in  table  1  below. 


1  The  purchase  of  certain  materials  or  parts  by  the  prime  contractor  from  vendors  in 
quantities  greater  than  those  needed  for  any  single  year  of  production  under  the  multiyear 
contract.  The  goal  is  to  minimize  the  costs  of  these  items  by  buying  in  larger  more 
economically  efficient  quantities  and  avoid  the  expenses  related  to  additional  production 
line  set-ups  and  terminations  that  an  annual  buy  approach  would  necessitate. 

2  Pub.  L.  No.  97-86,  §  909  (1981). 
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Table  1:  Statutory  Requirements  for  Multiyear  Procurement  Candidates 

Criteria 

Description 

Substantial  savings 

Candidate  programs  should  demonstrate  that  use  of  a 
multiyear  contract  will  result  in  substantial  savings  in  the  total 
estimated  costs  when  compared  to  the  use  of  a  series  of 
annual  contracts  for  the  same  procurement. 

Stability  of  the 
requirement 

Candidate  programs  should  demonstrate  that  the  minimum 
need  to  be  purchased  in  terms  of  total  quantity,  production 
rate,  and  procurement  rate  is  expected  to  be  substantially 
unchanged  during  the  multiyear  contract  period. 

Stability  of  funding 

Candidate  programs  should  have  a  reasonable  expectation 
that  sufficient  funding  will  be  requested  by  DOD  to  carry  out 
the  contract  and  avoid  cancellation  over  the  proposed 
multiyear  contract  period. 

Stable  design 

Candidate  programs  should  be  able  to  demonstrate  that  they 
have  technical  risks  that  are  not  excessive  over  the  multiyear 
period  and  that  the  items  procured  should  be  substantially 
unchanged  during  the  multiyear  period. 

Realistic  cost  estimates 

Candidate  programs  should  be  able  to  demonstrate  realistic 
estimates  of  contract  cost  and  projected  multiyear  savings  / 
cost  avoidance  through  the  use  of  a  multiyear  contract 
strategy. 

National  security 

Candidate  programs  should  be  able  to  show  that  the  use  of  a 
multiyear  contracting  strategy  will  promote  the  national 
security  interests  of  the  United  States  government. 

Source:  GAO  analysis  and  10  U.S.C.  2306b. 


Expected  costs  to  be  avoided  should  be  sufficient  to  offset  the  added  risk 
the  government  assumes  with  a  multiyear  contract  in  the  form  of  a 
cancellation  liability,  decreased  flexibility  in  future  funding  decisions  and 
any  erroneous  assumptions  in  the  estimates.  Immature,  volatile  programs 
and  those  at  risk  of  future  changes  should  not  be  proposed  as  MYP 
candidates  because  such  instability  puts  the  savings  attributed  to 
efficiencies  of  production  and  EOQ  buying  at  risk.  The  multiyear  approach 
should  be  reserved  for  established  production  operations  and  low  risk 
technology. 

In  submitting  candidates  for  multiyear  authorization  by  the  Congress,  the 
heads  of  the  respective  military  departments  vouch  that  each  program 
complies  with  the  criteria  in  table  1.  Additionally,  the  Secretary  of  Defense 
is  required  to  certify  to  Congress  that  the  current  future  years  defense 
program  fully  funds  the  support  costs  associated  with  the  multiyear 
contract  and  that  planned  production  will  not  be  less  than  the  minimum 
economic  rates  given  the  existing  tooling  and  facilities.  Multiyear 
contracts  historically  account  for  a  substantial  share  of  the  defense 
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procurement  dollar.  Figure  1  shows  that  DOD  has  budgeted  about  $10 
million  annually  for  multiyear  contracts  since  fiscal  year  2000,  accounting 
for  more  than  13  percent  of  DOD’s  total  budget  for  procurement  over  this 
time  frame.  Over  the  period,  the  general  trend  shows  an  increase  in  total 
defense  procurement,  but  multiyear  obligations  holding  fairly  steady, 
resulting  in  a  downward  trend  of  the  percentage  obligated  on  multiyear 
contracts.  For  2007,  the  large  increase  in  total  defense  procurement 
caused  a  drop  below  12  percent  obligated  on  multiyear. 


Figure  1:  Multiyear  Contract  Dollars  and  Percentage  Share  of  Total  Defense 
Procurement  Obligation  Authority 
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DOD’s  Practices  and 
Processes  for 
Justifying  and 
Approving  Multiyear 
Programs  Could  Be 
Improved 


The  statutory  criteria  for  a  multiyear  procurement  require  that  a  candidate 
program  make  realistic  cost  estimates,  expect  to  achieve  substantial 
savings,  and  provide  adequate  evidence  that  the  program  is  stable  in  terms 
of  funding,  requirements,  and  design.  Some  recent  programs  of 
questionable  stability  and  savings  submitted  to  the  Congress  for  multiyear 
authorization  raise  concerns  about  DOD’s  management  and  controls  for 
justifying  multiyear  candidates.  We  found  that  DOD  does  not  provide 
sufficient  guidance  and  direction  to  ensure  a  rigorous,  disciplined  process 
supported  by  adequate  empirical  data  for  preparing  and  reviewing 
multiyear  candidates.  This  increases  the  risk  of  poor  outcomes  and 
inappropriate,  unstable  programs  approved  for  multiyear  procurement. 


Questions  about  the 
Appropriateness  of 
Recently  Approved 
Programs 


Funding  and  Requirements 
Issues  on  F-22A  Raptor 
Program 


We  reviewed  DOD’s  multiyear  justification  data  submitted  in  recent 
defense  budgets  and,  in  particular,  examined  two  newly  approved 
programs — the  Air  Force’s  F-22A  fighter  and  the  joint  V-22  tilt  rotor 
aircraft.  The  F-22A  acquisition  has  had  a  turbulent  history  with  a  lengthy 
development  period,  major  cost  increases  and  quantity  decreases,  changes 
in  mission,  and  disagreements  within  DOD  about  the  total  number 
required.  The  Air  Force’s  submission  of  the  F-22A  for  multiyear 
procurement  generated  considerable  debate  over  its  merits  and  whether  it 
met  the  legal  and  business  conditions  conducive  to  success.  We  also 
examined  the  V-22  tilt  rotor  aircraft  multiyear  proposal,  another 
acquisition  program  with  a  turbulent  history  and  continuing  challenges. 

The  latest  restructure  of  the  F-22A  acquisition  occurred  in  December  2005. 
DOD  extended  production  2  years,  added  four  aircraft  and  $1  billion  in 
procurement  funds,  and  proposed  to  buy  the  final  60  aircraft  under  two 
separate  3-year  multiyear  contracts  for  the  airframes  and  engines. 
Multiyear  costs  and  savings  estimates  were  not  completed  in  time  for 
submission  with  the  fiscal  year  2007  defense  budget.  The  Air  Force  later 
submitted  the  completed  MYP  justification  package  with  estimated  total 
multiyear  costs  of  $8.7  billion  and  projected  savings  of  $235  million,  or  2.6 
percent,  compared  to  estimated  annual  contracts.  Multiyear  proponents 
cited  the  projected  total  dollar  savings  as  substantial  and  believed  there 
was  little  risk  that  the  remaining  60  aircraft  would  not  be  procured.  On  the 
other  hand,  multiyear  critics  argued  that  the  low  percent  of  savings 
predicted,  the  short  time  frame  for  accruing  savings,  and  the  program’s 
relatively  unstable  past  made  it  an  inappropriate  multiyear  candidate. 
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Stability  and  Production 
Concerns  about  the  V-22 
Osprey  Program 


In  prior  work,  we  determined  that  the  restructured  F-22A  program  was 
underfunded  and  questioned  whether  the  proposed  multiyear  strategy  met 
statutory  criteria.3  We  identified  concerns  about  savings,  funding, 
requirements,  and  design  stability  that  we  believed  needed  to  be  addressed 
before  the  multiyear  plan  could  be  justified.  For  example,  the  Air  Force 
did  not  fully  fund  the  multiyear  proposal  and  asked  for  incremental 
funding.  Also,  a  major  development  program  to  add  new  capabilities  and 
improve  reliability  of  the  F-22A  has  begun;  these  efforts  could  result  in 
future  design  modifications  which  may  require  retrofit  onto  aircraft 
purchased  under  the  multiyear  contract.  We  also  noted  that  having  only  a 
3-year  period  of  performance  at  the  end  of  production  limited  the  ability  to 
achieve  savings  normally  expected  under  multiyear  authority  such  as  EOQ 
buys  and  cost  reduction  initiatives  to  improve  manufacturing  efficiency. 

To  provide  for  EOQ  buys,  Congress  subsequently  added  $210  million  to 
the  F-22A  advance  procurement  budget.  In  authorizing  a  multiyear 
contract  for  the  F-22A,  Congress  specified  certain  conditions  to  be  met 
and  prohibited  the  use  of  incremental  funding.4  The  Secretary  of  Defense 
was  required  to  certify  that  all  statutory  requirements  have  been  met, 
including  the  determination  that  the  contract  will  result  in  substantial 
savings  of  the  total  anticipated  costs  of  carrying  out  the  program  through 
annual  contracts.  DOD  submitted  its  certifications  to  the  Congress  in  June 
2007  and  subsequently  awarded  the  F-22A  multiyear  contract  in  August 
2007.  For  the  fiscal  year  2009  budget  cycle,  the  Air  Force  continues  to 
propose  buying  more  aircraft  than  the  183  in  the  current  defense  plan. 

DOD  submitted  the  V-22  Osprey  for  multiyear  procurement  authorization 
in  the  fiscal  year  2007  President’s  Budget.  Officials  proposed  a  5-year 
multiyear  contract  to  acquire  185  aircraft  for  about  $10.1  billion.  Multiyear 
savings  were  projected  at  $435  million,  or  4.2  percent,  compared  to  the 
estimated  costs  for  annual  contracts. 

Ongoing  changes  in  quantity,  funding,  design,  and  concerns  about 
production  raise  questions  about  the  stability  of  this  program  and  its 


3  GAO,  Tactical  Aircraft:  Questions  Concerning  the  F-22A’s  Business  Case,  GAO-06-991T 
(Washington,  D.C.:  July  25,  2006)  and  GAO,  Tactical  Aircraft:  DOD  Should  Present  a  New 
F-22A  Business  Case  before  Making  Further  Investments,  GAO-06-455R  (Washington, 
D.C.:  June  20,  2006). 

4  John  Warner  National  Defense  Authorization  Act  for  Fiscal  Year  2007,  Pub.  L.  No.  109-365 
§134  (2006). 
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appropriateness  for  multiyear  contracting.  Subsequent  to  congressional 
authorization  for  multiyear  contracting,  DOD  reduced  its  planned 
procurements  quantity  from  185  to  167  due  to  service  funding  limitations 
with  DOD  cutting  the  proposed  procurement  request  for  fiscal  year  2009 
by  $234  million. 

Development  and  test  efforts  continue  with  a  number  of  design  changes 
under  review  to  address  serious  safety,  reliability,  and  performance 
problems.  The  program  office  has  aggressively  prioritized  these  issues 
and  is  making  improvements  to  the  V-22  platform  by  funding  engineering 
design  changes  for  the  correction  of  deficiencies.  One  such  deficiency  is 
leaking  hydraulic  fluid  causing  engine  compartment  fires.  Design  changes 
to  fix  this  deficiency  are  being  studied  and  implemented.  In  comments  on 
a  draft  of  this  report  the  program  office  stated  it  is  confident  that  these 
engineering  design  changes  will  address  the  hydraulic  leak  problems. 

To  date,  DOD  has  procured  111  aircraft  in  11  years.  Production  aircraft 
continue  to  be  conditionally  accepted  with  deviations  and  waivers. 
Engineering  investigations  to  fix  these  issues  are  not  complete  as  the 
program  continues  to  work  to  minimize  these  deviations  and  waivers. 

Even  so,  the  planned  production  rate  for  the  multiyear  period  is  twice  the 
current  fiscal  year  2007  production  rate  of  17  V-22s.  This  increase,  coupled 
with  design  and  production  problems,  raises  concerns  over  the 
contractor’s  ability  to  meet  such  a  demand.  DOD  reviews  and  assessments 
of  the  V-22  production  ramp  up  have  endorsed  the  increase  with  known 
risks  that  require  continued  management.  Officials  told  us  that  the 
supplier  base  should  be  able  to  meet  the  elevated  production  rate,  but 
expressed  concerns  about  the  availability  of  spare  parts  and  the 
challenges  in  managing  manufacturing  and  installation  at  three  different 
and  dispersed  facilities. 


Insufficient  Guidance  and 
Management  Direction  by 
DOD  to  Ensure  High 
Quality  Decisions 


The  statutory  criteria  in  section  2306b  of  title  10,  United  States  Code, 
establish  a  framework  for  limiting  multiyear  contracts  to  very  stable 
programs  that  appropriately  balance  risks  with  anticipated  savings.  DOD’s 
process  and  practices  for  justifying  multiyear  candidates,  however, 
provide  little  specific  guidance  on  the  meaning  and  application  of  the 
criteria,  a  process  that  allows  for  subjective  interpretations  about  how 
well  a  particular  program  meets  the  criteria.  Cost  and  savings  estimating 
techniques  and  data  also  vary  considerably  and  inadequate  records  are 
kept  to  document  decisions  and  supporting  reasons.  As  a  result,  costs, 
savings,  and  evidence  supporting  stability  is  not  consistently  prepared, 
reviewed,  and  documented. 
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Concerns  about  DOD’s 
Processes  and  Practices  for 
Justifying  Multiyear  Candidates 


The  statutory  criteria  are  general  in  tone  and  qualitative  by  nature  to 
provide  application  over  a  wide  range  of  programs.  The  regulations  DOD 
uses  to  implement  the  criteria  provide  contract  policies  and  establish  a 
process  for  developing  and  justifying  multiyear  candidates.5  Each 
candidate  program  prepares  a  budget  justification  package,  normally  for 
inclusion  with  the  annual  defense  budget  submission  that  provides  funding 
requirements  and  estimated  cost  savings  expected  under  a  multiyear 
contract  compared  to  the  estimated  costs  for  a  series  of  annual  contracts. 
The  justification  package  also  includes  short  statements  about  how 
programs  are  believed  to  meet  each  of  the  six  statutory  criteria.  Weapon 
system  acquisition  program  officials  prepare  the  package  for  subsequent 
reviews  by  military  service  acquisition  commands,  service  headquarters 
offices,  and  OSD  offices  responsible  for  program  policy  and  budget 
oversight.  Approved  candidates  are  submitted  to  the  Congress  for 
authorization. 

We  discussed  DOD’s  justification  process,  multiyear  contracting  policies, 
and  management  practices  with  OSD  and  military  service  officials  at  each 
organizational  level.  We  collected  historical  and  budget  data  on  approved 
multiyear  programs  since  fiscal  year  1982  and  tracked  more  recent 
multiyear  candidates  through  the  budget  process.  We  specifically 
examined  nine  major  programs  representing  each  military  department: 
three  had  completed  multiyear  contracts,  two  were  beginning  multiyear 
contracts,  and  four  programs  were  just  recently  authorized  by  the 
Congress.  Several  programs  have  been  approved  for  more  than  one 
multiyear  contract  and,  collectively,  these  programs  accounted  for  17  of 
the  59  approvals  granted  since  1996. 

We  found  that  DOD  provides  little  in  the  way  of  supplemental  guidance  to 
operationalize  the  statutory  criteria  by  amplifying  terms  such  as 
“reasonable,”  “substantial,”  and  “stable”  and  quantifying  where  possible  to 
provide  more  objectivity  and  rigor  to  the  multiyear  review  process. 
Guidance  for  the  most  part  restates  the  statutory  criteria  and  establishes 
formats  for  submitting  budget  justification  materials,  but  does  not  provide 
much  elucidation  for  interpreting  and  applying  the  criteria  and 
establishing  internal  evidence  standards  for  demonstrating  criteria  are 


5  Principally,  the  Federal  Acquisition  Regulation  (FAR),  and  the  Defense  Federal 
Acquisition  Regulation  Supplement  (DFARS),  and  DOD  Financial  Management  Regulation 
(FMR)  7000. 14-R. 
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met.6  From  our  review  of  justification  packages  and  our  discussions  with 
DOD  officials  responsible  for  generating  and  reviewing  multiyear 
justification  packages,  we  determined  that  reviewers  interpret  and  apply 
criteria  differently  and  that  the  methods  and  data  used  to  compute 
contract  costs  and  savings  and  providing  evidence  to  document  program 
stability  vary  in  quality  and  sophistication. 

For  example,  officials  we  talked  to  at  all  levels  of  the  review  process  had 
different  ideas  and  perspectives  on  what  constituted  substantial  savings 
when  applying  the  criterion.  An  official  in  the  Navy,  for  example,  expected 
programs  to  project  at  least  10  percent  savings,  but  would  consider 
candidates  under  that  level.  Some  Army  officials  wanted  to  see  10  percent 
savings  or  hundreds  of  millions  of  dollars.  An  OSD  official  applied  a  “rule 
of  thumb”  of  4  to  5  percent.  He  said  that  programs  under  that  amount 
would  typically  be  more  closely  evaluated  to  ensure  they  were  viable,  but 
that  programs  over  that  amount  would  generally  receive  a  less  detailed 
check  of  reasonableness  and  to  ensure  paperwork  requirements  were  met. 
An  Air  Force  official  told  us  that  a  5  percent  savings  level  should  be 
considered  the  floor  for  a  genuinely  viable  candidate  for  multiyear  and  a 
10  percent  savings  level  achievable,  although  he  cautioned  that  recent 
statutory  changes  to  eligible  cancellation  ceiling  costs  will  likely  have  a 
negative  impact  on  future  multiyear  savings. 

Further,  review  of  the  justification  packages  for  the  F-22A  and  the  V-22, 
both  submitted  in  the  same  fiscal  year  for  approval,  indicated  differences 
in  how  the  design  stability  criterion  was  applied.  Initial  operational 
capability,  an  important  milestone  for  stability,  had  been  declared  2  years 
prior  to  requesting  multiyear  authority  on  the  F-22A,  while  initial 
operational  capabilities  for  both  variants  of  the  V-22  are  not  scheduled  to 
be  achieved  until  after  multiyear  approval  was  granted,  at  least  2  years 
later  for  the  Navy’s  variant. 

Comparing  multiyear  savings  and  judging  reasonableness  of  the  estimates 
is  complicated  because  the  techniques  and  data  used  to  estimate  cost 
savings  can  vary  substantially  in  form,  sophistication,  and  detail.  For 
example,  for  its  first  multiyear  contract  proposal  on  the  C-17A,  the  Air 


6  At  the  Air  Force,  we  identified  a  guidebook  developed  in  the  1980s  that  provides  some 
details  and  examples  that  could  aid  officials  in  deciding  whether  a  multiyear  contract 
would  be  appropriate  and  in  developing  justification  materials.  We  found,  however,  that 
some  officials  were  either  unaware  of  this  or  mainly  used  the  FAR  and  DFAR  guidance. 
Officials  knew  of  no  comparable  document  in  the  Army,  Navy,  and  OSD. 
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Force  simply  used  the  prime  contractor’s  offer  to  save  5.5  percent,  a  figure 
that  was  considered  a  “management  challenge”  the  contractor  believed  it 
could  meet.  We  found  other  instances  where  it  was  unclear  what  data  was 
used  in  formulating  the  savings  estimates.  For  example,  the  Army’s  budget 
justifications  for  the  Bradley  Fighting  Vehicle  and  Abrams  Tank  estimated 
savings  at  5  percent  and  10  percent,  respectively,  and  provided  very 
sketchy  details  to  document  how  the  savings  estimates  were  derived  and 
compliance  with  the  stability  criteria. 

On  the  other  hand,  some  analyses  can  be  very  involved,  provide  a  range  of 
estimated  savings,  and  use  independent  third  parties  to  validate  data.  For 
example,  the  Navy’s  F-18E/F  cost  estimates  and  methodologies  were 
independently  verified  by  OSD’s  Cost  Analysis  Improvement  Group.  For 
the  F-22A  multiyear  proposal,  two  defense  research  centers  developed 
contract  costs  and  savings  estimates.  Several  very  different  methodologies 
were  employed,  including  cost  improvement  curves  based  on  historical 
actual  production  costs,  production  cost  estimate  models  for  single  year 
contracts  less  reductions  for  expected  savings,  and  summation  of  savings 
initiatives  from  the  prime  and  subcontractors.  These  studies  produced  a 
wide  range  of  potential  savings,  from  $235  million  to  $643  million. 

Also,  cost  and  savings  estimates  in  general  may  be  subject  to  biases  and 
other  factors  that  impact  their  fidelity  and  reliability.  Our  extensive  past 
body  of  work  on  DOD’s  major  acquisitions  suggests  that  to  gain  approval 
and  continued  funding  for  a  weapon  system,  the  acquisition  environment 
encourages  programs  to  submit  overoptimistic  estimates  about  a  weapon 
system’s  readiness  for  production  and  to  underestimate  its  costs.  Systems 
therefore  appear  more  affordable  from  an  investment  perspective  and  can 
fit  within  forecasts  of  available  funds.  These  circumstances  invariably  lead 
to  acquisition  programs  costing  substantially  more  than  originally 
estimated.7  Furthermore,  prior  reports  have  discussed  the  importance  of 
using  present  value  analysis  to  account  for  the  time  value  of  money  when 
evaluating  and  comparing  costs  of  alternative  annual  and  multiyear 
contracts.  The  timing  of  government  expenditures  is  expected  to  differ 
with  a  multiyear  contract  expected  to  have  relatively  more  up-front  costs 
(to  fund  EOQ  for  example)  and  lower  costs  in  the  outyears  compared  to  a 


'  GAO,  DOD  Acquisition  Outcomes:  A  Case  for  Change,  GAO-06-257T  (Washington,  D.C.: 
Nov.  15,  2005)  and  GAO,  Weapons  Acquisition:  A  Rare  Opportunity  for  Lasting  Change, 
GAO/N SIAD-93- 1 5  (Washington,  D.C.:  Dec.  1,  1992). 
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Documentation  and  Record- 
Keeping  Deficiencies 


series  of  annual  contracts.8  Although  the  justification  packages  are 
required  to  have  a  present  value  analysis  of  the  savings  estimates, 
according  to  an  OSD  official  and  a  defense  research  study,  the  cost  savings 
estimates  in  then-year  dollars  are  the  primary  estimates  used  in  making 
cost  decisions.  Our  review  of  the  justification  packages  appear  to  confirm 
this  because  the  section  of  the  multiyear  exhibit  highlighting  the  benefit  to 
the  government  contained  only  the  then-year  dollar  estimate  rather  than 
the  present  value  estimate. 

Through  our  discussions  with  officials  and  inspection  of  records,  we 
determined  that  DOD’s  review  process  for  the  multiyear  justification 
budget  packages  does  not  adequately  capture  important  information  and 
events  to  document  decisions  and  help  ensure  that  consistent  and  reliable 
determinations  are  made  regarding  multiyear  criteria.  Once  approved, 

OSD  officials  stated  few  records  are  kept  on  multiyear  programs  regarding 
how  they  determined  whether  multiyear  candidates  met  the  six  statutory 
criteria.  According  to  OSD  and  service  officials,  much  of  the  discussion  on 
a  program  proposed  for  multiyear  should  have  already  taken  place  during 
regular  executive-level  reviews  of  major  weapon  systems  and  been  agreed 
upon  before  the  multiyear  justification  package  is  reviewed  for  submission 
in  the  budget  request.  Review  of  the  justification  package  then  essentially 
becomes  a  paperwork  formality  rarely  involving  any  surprises.  Also,  we 
found  programs  can  be  proposed  “out  of  cycle”  with  the  President’s 
Budget  submission — as  in  the  case  of  the  F-22A,  and  may  not  be  included 
in  the  budget  details  that  could  affect  the  review  path  the  multiyear 
candidate  takes  to  obtain  approval.  Without  maintaining  records  that 
document  decisions  and  the  data  supporting  them,  it  is  difficult  to  ensure 
the  quality  and  comprehensiveness  of  stakeholder  reviews  based  on  the 
criteria,  fidelity  of  the  data  used,  and  supporting  rationales  for  decisions. 

Finally,  officials  at  every  level  of  the  multiyear  justification  process — from 
program  offices,  through  higher  headquarters,  and  on  to  primary  OSD 
action  offices — indicated  that  they  recently  were  appointed  to  their 
current  positions  or  the  person  responsible  during  the  multiyear 
justification  process  was  no  longer  in  that  position.  We  believe  this 
contributed  to  “knowledge  gaps,”  historical  record-keeping  deficiencies, 


8  GAO,  An  Assessment  of  the  Army’s  Multiple  Launch  Rocket  System  Multiyear  Contract, 
(GAO/NSIAD-86-5  Oct.  28,  1985);  GAO  Procurement:  An  Assessment  of  the  Air  Force’s  F- 
16  Aircraft  Multiyear  Co?itract  (GAO/N SAID-86-38,  February  1986);  and  GAO 
Procurement:  Assessment  of  DOD’  Multiyear  Contract  Candidates 
(GAO/NSAID-87-202-BR  August  1987). 
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and  differences  in  interpretation  and  application  of  multiyear  decision 
criteria.  Turnover  implies  loss  of  experience  and  corporate  knowledge;  in 
this  environment,  improved  and  more  definitive  guidance  and  retention  of 
comprehensive  historical  records  is  even  more  important. 


Actual  Results  on 
Some  Completed 
Contracts  Were  Much 
Different  Than 
Predicted  in  Budget 
Justifications 


Implementing  the  statutory  criteria  requires  realistic  estimates  of 
multiyear  and  annual  contract  costs.  This  requirement  provides  fidelity  to 
savings  projections  and  allows  for  accurate  estimates  of  funding 
requirements  over  the  life  of  the  multiyear  contract.  We  reviewed  cost 
performance  and  results  on  completed  multiyear  contracts  for  the  Air 
Force’s  C-17A  Globemaster  transport,  the  Navy’s  F/A-18E/F  Super  Hornet 
fighter,  and  the  Army’s  Apache  Longbow  helicopter.9  Although  the  precise 
impact  of  multiyear  contracting  is  difficult  to  determine,  our  analysis 
shows  that  these  programs — presumably  approved  based  on  their 
demonstrated  stability — experienced  substantial  changes  during  contract 
execution.  These  changes  significantly  increased  unit  costs  and  drove  up 
total  funding  requirements  much  beyond  the  estimates  submitted  to 
Congress  in  the  budget  justification  materials.  Each  was  also  impacted  by 
contract  provisions  and  changes  in  business  conditions. 


Cost  Increases  Lead  to 
Questions  about  Realism 
of  Budget  Estimates  Used 
to  Justify  Multiyear 


We  found  that  unit  cost  growth  on  these  programs  ranged  from  10  to  30 
percent  more  than  projected  by  the  budget  justification  data.  Table  2 
shows  the  growth  in  unit  and  total  contract  costs.  We  also  found  that,  for 
two  of  the  three  programs,  actual  multiyear  contract  costs  exceeded  the 
original  estimates  for  annual  contract  costs.  The  third  program,  the  F/A- 
18E/F,  came  in  below  annual  estimates,  but  also  bought  fewer  systems 
than  planned.  This  reduction  in  quantity  would  have  also  likely  decreased 
annual  costs  had  that  alternative  been  selected. 


9  For  comparability  purposes  and  availability  of  data,  we  selected  these  three  aircraft 
programs,  each  with  recently  completed  multiyear  contracts  and  ongoing  follow-on 
multiyear  contracts.  Our  findings  cannot  be  extended  to  multiyear  contracting  in  total,  but 
do  provide  illustrations  of  what  can  happen  during  execution  of  a  multiyear  contract. 
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Table  2:  Unit  Cost  Growth  for  Three  Multiyear  Contracts 


Weapon 

system 

Original 

savings 

projection 

Original 

quantity 

Final 

quantity 

Original  MYP 
contract 
estimated 

Original  unit 
cost 

Final  MYP 
contract 
value 

Final  unit 
cost 

Unit  cost 
increase 
(percent) 

C-17A 

5.5% 

80 

80 

$14,354.0 

$179.43 

$16,614.0 

$207.68 

15.7% 

F/A-18E/F 

7.4 

222 

210 

8,840.8 

39.82 

9,221.8 

43.91 

10.3 

Apache 

7.6 

232 

232 

1,596.4 

6.88 

2,078.8 

8.96 

30.2 

Source:  GAO  analysis  of  MYP  justification  packages  and  selected  acquisition  reports. 


Substantial  cost  increases  for  completed  multiyear  contracts  on  the  three 
programs  meant  that  Congress  had  to  eventually  provide  considerably 
more  funding  than  originally  budgeted.  We  do  not  know  how  cost  growth 
affected  the  level  of  savings  achieved,  if  any,  because  we  do  not  know  how 
an  alternative  series  of  annual  contracts  would  have  fared.  In  comments 
on  a  draft  of  this  report,  DOD  officials  stressed  that  cost  growth  due  to 
labor  and  material  price  escalation  under  a  multiyear  contract  would  likely 
have  also  occurred  under  an  alternative  series  of  annual  contracts.  The 
final  MYP  contract  values  in  table  3  also  include  price  increases  resulting 
from  engineering  design  changes  made  to  the  baseline  weapon  system. 
Although  these  factors  may  limit  the  ability  to  make  inferences  about  the 
level  of  savings  achieved,  a  case  could  be  made  that  multiyear  savings  and 
costs  did  not  materialize  as  presented  in  the  multiyear  justification 
materials. 

As  discussed  earlier,  our  past  body  work  suggests  that  defense  acquisition 
programs  are  prone  to  underestimating  costs  and  overstating  readiness. 
While  this  tendency  would  apply  to  annual  as  well  as  multiyear  contracts, 
it  is  arguably  more  problematic  for  multiyear  contracts  because  of  the 
government’s  increased  exposure  to  risk  over  multiple  years.  DOD 
officials  agreed  with  us  that  multiyear  contracting  should  be  held  to  a  high 
standard  because  of  its  special  requirements,  funding  commitments,  and 
risks. 


Impacts  on  Cost  and 
Performance  from 
Contract  Provisions  and 
Other  Factors 


We  also  collected  information  on  multiyear  contract  provisions  for  the 
three  programs  with  completed  multiyear  contracts  and  for  the  recently 
awarded  F-22A  contract.  Each  of  these  programs  awarded  a  fixed-price 
contract  for  the  multiyear  procurement,  but  they  were  not  always  firm- 
fixed-price  contracts,  which  typically  entail  the  least  risk  to  the 
government.  The  multiyear  contracts  contained  standard  provisions  that 
provided  flexibility  to  increase  or  decrease  costs  based  on  inflation,  labor 
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rate  changes,  and/or  material  cost  fluctuations.10  The  multiyear  contracts 
also  included  provisions  for  early  cancellation,  quantity  variations,  and/or 
design  changes.  In  some  cases,  the  government  waived  provisions  for  cost 
and  pricing  data,  which  according  to  officials  decreased  the  government’s 
insight.  Figure  2  below  shows  the  frequency  of  the  various  contract 
provisions  in  our  case  studies. 


Figure  2:  Contract  Provisions  That  Impacted  Case  Study  Multiyear  Procurements 


C-17A 
MYP  1 

C-17A 
MYP  II 

F/A-18E/F 
MYP  1 

F/A-18E/F 
MYP  II 

Apache 
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MYP  II 

F-22A 
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adjustments 

/ 
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Cost/pricing  data 
waivers 

/ 

/ 

/ 

/ 

y 

Cancellation 
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(unfunded) 

/ 

/ 

/ 
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Source:  DOD  contract  file  information  and  officials. 


Case  Studies  Provide  The  three  case  studies  provide  insight  into  multiyear  contracting 

Details  on  Meeting  Cost  expectations  and  realities  and  the  internal  and  external  factors  that 
and  Program  Expectations  affected  actual  execution.  Each  case  raises  some  questions  about  the 

accuracy  of  cost  and  savings  estimates  used  to  justify  multiyear 
procurement  and  the  degree  to  which  requirements,  design,  and  funding 
were  stable. 

C-17A  Globemaster:  The  estimated  savings  for  the  first  C-17A  Globemaster  multiyear  contract 

Risky  Strategy  Affected  was  simply  a  percentage  amount  submitted  by  the  contractor.  For  the 

Multiyear  Funding  second  multiyear  contract,  Air  Force  officials  assumed  the  same  percent 

Stability  of  savings  and  added  additional  savings  based  on  the  use  of  a 

controversial  funding  strategy.  This  strategy  relied  on  incremental  funding, 
advanced  buys  of  parts,  and  large  potential  cancellation  liability  to 
maintain  a  production  schedule  of  15  aircraft  per  year  even  though  all  of 


10  The  Federal  Acquisition  Regulation  (FAR)  contains  examples  of  standard  economic  price 
adjustment  clauses  at  FAR  §  52.216-2  through  4  and  value  engineering  at  52.248-1  through  3. 


Page  17 


GAO-08-298  Defense  Acquisitions 


these  aircraft  had  not  been  fully  funded.  The  unfunded  liability  to  the  Air 
Force  had  the  contract  been  canceled  eventually  grew  to  $1.5  billion. 
Concerned  that  this  incremental  funding  strategy  violated  DOD’s  full¬ 
funding  policy  and  could  potentially  violate  the  Anti-Deficiency  Act, 
Congress  increased  C-17A  procurement  funding  a  total  of  $745  million  in 
fiscal  years  2003  and  2005  to  fully  fund  all  aircraft.  In  annual  DOD 
appropriations  acts,  Congress  also  has  prohibited  incremental  funding  of 
multiyear  contracts.11 

In  addition,  the  costs  for  both  C-17A  multiyear  contracts  were  affected  by 
economic  price  adjustments.  On  the  first  multiyear,  overhead  costs  were 
significantly  increased  as  a  result  of  a  merger  between  two  major  defense 
contractors.  The  Air  Force  subsequently  paid  $150  million  to  cover  cost 
increases  resulting  from  the  merger  and  another  $50  million  to  remove  the 
clause  from  the  contract.  By  the  end  of  the  multiyear  contract,  unit  costs 
had  increased  15  percent.  The  multiyear  justification  materials  submitted 
to  Congress  supported  a  plan  to  buy  80  aircraft  at  an  average  cost  of  $179 
million;  instead  the  Air  Force  eventually  paid  about  $207  million  per 
aircraft.  During  the  second  multiyear  contract,  the  contractor  made  large 
contributions  to  its  pension  fund,  which  triggered  the  price  adjustment 
clause  and  resulted  in  a  potential  cost  increase  of  over  $530  million.  The 
Air  Force  is  in  the  process  of  restructuring  the  contract  to  reduce  this 
amount. 

The  Air  Force  also  awarded  two  multiyear  contracts  for  the  C-17A  engine, 
the  FI  17.  These  procurements  appear  to  have  been  successful  with 
demonstrated  stability  during  the  multiyear  period  and  price  breaks  based 
on  the  multiyear  contract.  The  FI  17  engine  is  a  commercially  available 
engine  with  a  stable  design  and  manufacturing  process.  There  were  no 
engineering  or  design  changes;  no  advanced  procurement  or  EOQ 
requirements;  and  no  cancellation  ceilings  associated  with  either  contract. 
The  only  potential  cost  risk  was  the  economic  price  adjustment  clause,  but 
officials  stated  that  the  actual  adjustments  were  not  exclusively  in  the 
contractor’s  favor.  According  to  the  program  office,  the  first  multiyear 
contract  resulted  in  a  savings  of  10  percent,  more  than  it  had  originally 
expected.  Savings  from  the  second  multiyear  contract  are  consistent  with 
the  original  estimate. 


11  See  Department  of  Defense  Appropriations  Act,  2005,  Pub.  L.  No.  108-287,  §  8008  (2004), 
and  similar  provisions  in  subsequent  DOD  appropriations  acts. 
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F/A-18E/F  Super  Hornet: 
Design  and  Requirement 
Changes  Impact  Costs 


For  its  first  F/A-18E/F  multiyear  contract,  the  Navy  did  not  award  a  firm- 
fixed-price  contract  because  the  program  was  early  in  the  production 
phase  and  there  were  still  ongoing  design  development  efforts  on  the 
airframe.  During  the  contract  period,  the  economic  price  adjustment 
clause  resulted  in  the  Navy  paying  an  additional  $378  million  because  of 
labor  rate  and  material  cost  increases.  The  first  multiyear  contract  also 
included  a  variation-in-quantity  clause  that  permitted  an  upward  or 
downward  adjustment  of  six  aircraft  per  year.  Annual  quantities  and  the 
specific  mix  of  buys  between  the  two  models  changed  more  than  once 
during  the  multiyear  period.  By  the  end  of  the  multiyear  contract,  the 
number  of  aircraft  procured  had  dropped  from  222  to  210  aircraft  and  the 
average  unit  costs  had  increased  by  10  percent,  compared  to  the  budget 
estimates. 

In  a  May  2000  report,  we  had  questioned  whether  the  Navy  was  ready  to 
enter  into  its  first  multiyear  contract  for  full-rate  production.  Deficiencies 
identified  during  operational  testing  had  not  been  corrected,  and  to  avoid 
costly  retrofitting  and  redesign,  we  believed  that  corrections  should  be 
made  and  tested  before  entering  into  the  contract.12 

The  Navy  proposed  buying  another  210  Super  Hornets  on  a  second 
multiyear  contract,  but  later  changed  the  requirement  to  procure  154 
Super  Hornets  and  56  of  the  new  E/A-18G  Growler,  an  electronic  attack 
variant  still  in  product  development.  The  Navy’s  total  requirement  for  the 
Super  Hornet  had  been  reduced,  and  the  new  Growler  was  needed  to 
replace  aging  EA-6B  aircraft  in  the  electronic  attack  mission.  The  follow- 
on  multiyear  also  included  a  variation-in-quantity  clause,  but  this  time  it 
only  covered  upward  adjustments.  Multiyear  costs  and  funding  were 
further  impacted  by  the  economic  price  adjustment  clauses.  As  in  the 
C-17A’s  case,  the  F/A-18E/F  multiyear  contract  was  affected  by  the 
contractor’s  large  pension  fund  contribution.  The  Navy  estimated  that  it 
could  have  been  obligated  to  pay  over  $1  billion,  which  is  nearly  the  same 
as  the  amount  of  cost  savings  originally  estimated  to  justify  the  multiyear 
contract  award.  However,  the  Navy  renegotiated  the  terms  of  the  clause 
and  restructured  the  contract  to  bring  the  price  adjustment  down  to  about 
$152  million. 


12  GAO,  Defense  Acquisitions:  F/A-18E/F  Aircraft  Does  Not  Meet  All  Criteria  for 
Multiyear  Procurement,  GAO/NSIAD-OO-158  (Washington,  D.C.:  May  26,  2000). 
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Longbow  Apache:  The  Army’s  Longbow  Apache  helicopter  experienced  significant  cost 

Modifications  and  Unplanned  increases  during  both  its  multiyear  procurements.  Army  officials  stated 
Work  Increased  Unit  Costs  that  increases  were  largely  because  of  aircraft  modifications  and 

unplanned  work.  These  modifications  included  a  voice  data  recorder  and 
an  improved  rotor  blade  assembly  that  would  enhance  operational  safety. 
Contract  costs  were  also  increased  by  additional  unplanned  work. 

Program  officials  explained  that  it  is  very  difficult  to  predict  the  condition 
of  fielded  aircraft  when  they  return  to  be  upgraded  or  remanufactured. 
Along  with  normal  wear  and  tear,  many  operational  aircraft  were  returned 
with  extensive  corrosion  and  battle  damage;  others  had  been  cannibalized 
for  parts.  Remanufacturing  these  aircraft  required  significantly  more  effort 
and  funding  than  originally  planned.  By  the  end  of  the  first  multiyear 
contract,  the  Apache’s  average  unit  cost  had  increased  by  30  percent;  at 
the  end  of  the  second  multiyear,  these  costs  had  increased  by  25  percent. 

While  it  may  be  difficult  to  predict  unusual  wear  and  tear  on  a  system  and 
it  is  common  to  incorporate  new  modifications  over  time,  it  is  especially 
problematic  to  roll  these  costs  into  the  multiyear  contract  that  had  been 
assumed  stable  and  that  had  been  justified  based  on  initial  cost  estimates 
without  these  new  add-ons.  OSD  cost  analysts  are  studying  this  issue  to 
determine  the  proper  accounting  for  modifications  under  multiyear 
contracting. 


Estimated  Savings  for 
Candidate  Programs 
Are  Trending 
Downward 


The  amount  and  percentage  of  savings  expected  from  a  multiyear  contract 
compared  to  a  series  of  estimated  annual  contracts  is  the  most  visible  and 
perhaps  the  most  critical  criteria  in  the  eyes  of  many  stakeholders.  The 
savings  requirement  in  definition  and  application  has  evolved  over  the 
years.  A  threshold  level  of  10  percent  savings  emphasized  during  the  1980s 
was  eliminated  and  replaced  with  a  nonquantifiable  requirement  for 
“substantial  savings”  since  fiscal  year  1991,  allowing  wide  flexibility  in  its 
interpretation.13  Although  a  direct  causal  link  is  not  demonstrated,  our 
analysis  of  multiyear  programs  approved  by  Congress  shows  that 
estimated  savings  were  on  average  higher  before  the  substantial  savings 
requirement  than  after.  Other  factors — lower  quantities  of  modern  systems 
being  procured,  stricter  termination  liability  allowances,  and  contraction 
in  the  defense  industrial  base — may  also  contribute  to  decreased 


13  The  Conference  Report  for  the  National  Defense  Authorization  Act  for  Fiscal  Year  2008 
indicates  a  preference  for  a  10  percent  estimated  savings  level  for  future  multiyear 
candidates.  See  H.R.  Conf.  Rep.  No.  110-447,  at  951  (2007). 
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estimated  savings  for  current  and  future  systems  by  lessening  the  benefits 
from  large  quantity  buys  and  efficient  production  rates,  key  attributes  of 
multiyear  contracts. 


Multiyear  Savings 
Percentages  Were 
Generally  Lower  after 
Changes  in  the  Statutory 
Savings  Criteria 


When  Congress  codified  the  authority  for  multiyear  procurement 
contracting  in  December  1981, 14  there  was  no  specific  savings  criterion  in 
the  law  that  candidate  programs  had  to  meet.  However,  the  impetus 
behind  multiyear  was  provided  by  DOD  studies  at  the  time  predicting 
savings  averaging  10  to  20  percent.  The  10  percent  figure  became  a  savings 
benchmark  for  decision-makers  in  the  early  1980s  when  judging  the  merits 
of  candidate  programs.  In  the  late  1980s,  this  benchmark  became  a 
threshold  requirement  for  many  candidate  programs  as  Congress  began 
stipulating  a  10  to  12  percent  savings  amount  in  annual  defense 
authorization  acts  for  selected  programs. 


In  November  1989,  Congress  decided  to  codify  the  10  percent  savings 
requirement  and  other  conditions  and  limitations  previously  imposed  on 
an  annual  basis.15  However,  a  year  later,  that  threshold  was  struck  from  the 
U.S.  Code,  and  the  requirement  for  “substantial  savings”  was  substituted 
after  DOD  had  argued  that  a  rigid  threshold  limited  the  potential  for 
savings  on  stable,  low  risk  programs  projecting  lesser  savings  amounts.16 
This  substantial  savings  requirement  has  remained  unchanged  since 
November  1990,  with  specific  savings  requirements  stipulated  in  annual 
legislation  for  two  candidate  programs  approved  by  Congress  during  this 
time.  Figure  3  illustrates  the  evolution  of  the  savings  requirement. 


14  Department  of  Defense  Authorization  Act,  1982,  Pub.  L.  No.  97-86,  §  909  (1981). 

15  National  Defense  Authorization  Act  for  Fiscal  Years  1990  and  1991,  Pub.  L.  No.  101-189, 
§  805  (1989). 

16  National  Defense  Authorization  Act  for  Fiscal  Year  1991,  Pub.  L.  No.  101-510,  §  808 
(1990). 
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Figure  3:  Legislative  History  Timeline  for  Multiyear  Savings  Requirement 


Source:  GAO  analysis. 


Our  analysis  of  estimated  savings  for  approved  multiyear  programs 
determined  that  the  average  savings  level  trended  lower  after  the 
substantial  savings  criterion  was  adopted.  Although  programs  have  been 
approved  during  both  eras  over  a  wide  range  of  savings  below  and  above 
10  percent,  the  change  in  the  law  provided  decision  makers  the  flexibility 
to  propose  and  approve  candidate  programs  since  fiscal  year  1991  with 
lower  savings  estimates  on  average  compared  to  the  1980s.  As  previously 
discussed,  an  unofficial  rule  of  thumb  for  savings  normally  expected  is 
now  down  to  4  to  5  percent. 


Figure  4  illustrates  the  range  of  savings  and  general  trend  as  the  savings 
requirement  evolved.  It  suggests  that  a  shift  did  occur  since  the  change  to 
substantial  savings  in  1991.  A  larger  proportion  of  multiyear  programs  with 
estimated  savings  of  less  than  10  percent  were  approved  after  1991.  On  the 
other  hand,  candidate  programs  approved  prior  to  1991  show  a  larger 
proportion  of  savings  of  10  percent  or  more.  This  finding  is  supported  by  a 
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defense  consultant  study  that  calculated  estimated  savings  averaged  13 
percent  for  candidate  programs  from  1982  to  1989. 17 


Figure  4:  Range  of  Estimated  Savings  for  Multiyear  Programs 

Number  of  MYP  programs 
40 


35  35 


Less  5-10  10-15a  Greater 

than  5  than  15 

Estimated  savings  percentage 


1991 -present  (after  the  “substantial  savings”  requirement) 

1982-1990  (before  the  change  in  the  savings  requirement) 

Source:  GAO  analysis  of  DOD  budget  and  prior  GAO  work. 

“The  ‘10-15’  category  includes  those  MYP  candidates  with  estimated  savings  of  exactly  10  percent. 


Other  Factors  Affecting  In  addition  to  the  revised  savings  requirement,  other  factors  may  also  be 

Estimated  Savings  impacting  the  level  of  savings  for  current  and  future  multiyear  programs: 

•  Smaller  quantities  procured.  The  higher  cost  for  modern  weapon 
systems  and  changes  in  required  force  structure  has  resulted  in 
generally  smaller  procurement  quantities  for  new  systems  compared  to 
predecessor  systems.  Smaller  quantities  of  systems  bought  under 
multiyear  contracts  may  not  provide  the  same  opportunity  to  achieve 
savings  through  such  mechanisms  as  EOQ  buys  of  parts  and  materials. 
Past  multiyears,  on  the  F-16  Falcon  aircraft  and  Black  Hawk 


17  Institute  for  Defense  Analyses,  F/A-22  Independent  Cost  Estimate  (August  2005). 
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helicopters,  for  example,  procured  hundreds  of  systems.  In  contrast, 
the  more  recent  multiyear  procurement  of  the  F/A-18E/F  Super  Hornet 
procured  less  than  half  the  F-16  quantity.  Analysis  of  the  data  on  MYP 
candidate  programs  that  we  collected  and  summarized  in  table  3  below 
showed  that  the  median  multiyear  procurement  quantity  for  aircraft 
MYP  candidate  programs  declined  over  40  percent  from  the  1980s  to 
the  present  era  with  a  concurrent  decline  in  average  savings.  With 
fewer  aircraft  procured  during  the  multiyear  period,  savings  from 
economic  buys  and  optimized  production  is  typically  smaller. 


Table  3:  Characteristics  of  Aircraft  MYP  Candidate  Programs 

1980s  vs. 

1990s-2000s 

Median  MYP  quantity 

252  units 

140  units 

Median  MYP  savings  percentage 

10.7% 

7.2% 

Source:  GAO  analysis  of  DOD  budget  justification  materials  and  prior  GAO  work. 


•  Cancellation  liability  changes.  If  a  multiyear  contract  is  cancelled,  the 
government  is  liable  for  reimbursing  the  contractor  for  its  incurred 
costs  up  to  a  negotiated  cancellation  ceiling  typically  in  the  contract. 
Until  recently,  DOD  has  been  able  to  include  both  recurring  and 
nonrecurring  costs.18  Multiyear  contracts  awarded  during  the  past  25 
years  have  included  a  cancellation  liability,  but  the  cancellation  ceiling 
for  the  C-17A  multiyear  contract  awarded  in  2002  was  considered  very 
large  with  a  potential  liability  of  more  than  $1.5  billion.  Some  members 
of  Congress  were  concerned  by  its  size  and  concluded  that  this  large 
liability  inappropriately  committed  the  government  to  a  production 
schedule  for  which  funding  had  not  been  appropriated.  As  a  result,  for 
the  past  several  years  Congress  has  limited  the  cancellation  liability  for 
multiyear  contracts  to  nonrecurring  costs  only.19  Some  DOD  officials 
expect  this  change  to  limit  savings  on  current  and  future  multiyear 
contracts  as  contractors  choose  not  to  bear  financial  risks  previously 
borne  by  the  government.  A  major  source  of  multiyear  savings  is  the 
EOQ  buys,  and  without  including  these  kinds  of  recurring  costs  in  the 
cancellation  ceiling,  fewer  contractors  may  buy  supplies  and  materials 


18  Recurring  costs  are  production  costs,  such  as  labor  and  materials  that  vary  with  the 
quantity  being  produced.  Nonrecurring  costs  are  fixed  expenses  that  do  not  vary  with  the 
quantity,  such  as  capital  investments  in  facilities,  equipment,  and  tooling. 

19  See  Department  of  Defense  Appropriations  Act,  2005,  Pub.  L.  No.  108-287,  §  8008  (2004) 
and  similar  provisions  in  subsequent  DOD  appropriations  acts. 
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up  front  in  bulk  to  limit  their  risks  should  the  multiyear  contract  be 
cancelled  early. 

•  Declining  competition.  Some  DOD  and  defense  research  center 

officials  believe  that  the  consolidation  and  contraction  of  the  defense 
industry  and  resulting  decline  in  competition  and  contraction  among 
vendors  and  suppliers,  make  it  harder  to  wring  savings  from  EOQ  buys. 
For  example,  an  OSD  official  stated  that  because  the  F-22A  was 
originally  designed  well  over  a  decade  ago,  it  is  now  experiencing 
diminishing  manufacturing  sources  on  many  components  as  well  as 
parts  and  equipment  obsolescence.  Similarly,  a  defense  research  center 
official  believes  that  diminishing  manufacturing  sources  negatively 
impacted  the  multiyear  savings  potential  for  the  F-22A. 


Multiyear  Contracting 
Results  Are  Uncertain 


DOD  does  not  perform  post  contract  assessments  of  completed  multiyear 
contracts  to  validate  actual  results  and  determine  cost  effectiveness  of  its 
investments.  Some  prior  studies,  including  GAO  work,  provide  some 
limited,  but  inconclusive  insights  into  multiyear  results  and  benefits.  OSD 
and  the  military  services  do  not  maintain  adequate,  comprehensive 
records  on  historical  and  current  multiyear  contracts  that  could  facilitate 
better  tracking  and  provide  perspective  for  future  multiyear  efforts. 


DOD  Does  Not  Have  a 
Formal  Process  for 
Assessing  and  Tracking 
Multiyear  Contract  Results 


DOD  does  not  have  a  formal  process  for  assessing  the  cost  and 
performance  of  completed  multiyear  contracts.  According  to  DOD 
officials,  once  a  program  is  approved  for  multiyear,  they  do  not  track 
subsequent  performance  nor  validate  actual  results  against  expectations 
established  in  the  budget  justification  submissions.  Also,  they  do  not  have 
an  official  method  to  capture  and  share  lessons  learned  with  other 
programs  considering  multiyear  contracts  that  could  improve  prospects 
for  successful  outcomes. 


However,  we  did  find  that  individual  program  offices  may  make  efforts  to 
ascertain  benefits  and  learn  lessons  that  can  be  applied  to  future  multiyear 
submissions,  but  it  appears  that  these  efforts  are  isolated.  Based  on  prior 
multiyear  experiences,  F/A-18E/F  Super  Hornet  program  officials  decided 
to  require  cost  and  pricing  data  to  better  inform  cost  estimates  for  their 
next  multiyear  application.  Also,  they  based  expected  savings  on  cost 
reduction  initiatives  rather  than  EOQ  buys  because  they  believed  these 
initiatives  had  a  better  return  on  investment.  Similarly,  the  results  of  our 
case  studies  discussed  earlier  demonstrate  that  assessing  actual  results 
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can  glean  valuable  information  about  contract  costs  and  performance  that 
can  be  used  to  improve  future  multiyear  outcomes. 


Prior  Studies  of  Multiyear 
Cost  Effectiveness  Show 
Limited  and  Inconclusive 
Results 


Some  attempts  to  assess  historical  multiyear  performance  have  been 
made,  but  validating  actual  savings  is  elusive.  According  to  DOD  and 
defense  research  center  officials  as  well  as  the  studies  they  conducted, 
calculating  the  actual  cost  savings  from  the  use  of  a  multiyear  contract  and 
comparing  results  to  original  expectations  is  very  problematic  for  several 
reasons:  (1)  multiyear  cost  and  other  program  data  is  unavailable;  (2)  lack 
of  comparable  data  on  costs  of  annual  contracts;  and  (3)  original 
assumptions  change  from  the  justification  package,  such  as  design 
modifications  and  variations  in  buy  quantities,  labor,  and  material  rates. 


Recent  studies  by  two  defense  research  centers  attempted  to  gain  a 
historical  perspective  on  actual  multiyear  savings  achieved  for  past 
contracts  in  order  to  provide  context  as  to  the  relative  level  of  expected 
savings  for  the  F-22A  multiyear  proposal.  In  reports,  both  centers  noted 
that  the  government  does  not  collect  or  save  data  needed  to  do  a  detailed 
analysis  and  that,  once  programs  are  approved  and  implemented, 
important  assumptions  on  which  original  savings  estimates  were  based 
often  changed. 


In  particular,  since  a  series  of  annual  contracts  are  not  executed  for  the 
same  procurement  once  a  multiyear  strategy  has  been  adopted, 
comparisons  of  actual  multiyear  costs  can  only  be  compared  to 
hypothetical  estimates  of  annual  contracts.  Further,  changes  that  occur  in 
a  multiyear  contract  environment  can  also  occur  in  an  annual  contract 
environment,  and  the  exact  effect  on  the  actual  costs  for  the  annual 
contracts  is  unknown.  In  reviewing  the  literature,  one  center  noted  that  it 
found  “very  few  examples  of  serious  and  methodologically  credible 
attempts  to  validate  claimed  savings  and  savings  percentages  after  the  fact 
once  programs  had  been  completed,”  and  none  that  produced,  in  the 
center’s  opinion,  definitive  findings.20 


In  the  case  of  the  F/A-18E/F  program,  researchers  from  one  center  felt 
they  had  more  data  available  on  this  multiyear  contract  than  for  others, 
and  they  attempted  to  validate  actual  savings.  They  concluded  that  while 


20  RAND  Corporation,  National  Defense  Research  Institute,  F-22A  Multiyear  Procurement 
Program:  An  Assessment  of  Cost  Savings,  2007. 
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the  available  data  supported  savings  in  the  neighborhood  of  original 
estimates,  a  definitive  answer  would  require  a  more  detailed  analysis  of 
EOQ  data  and  cost  reduction  initiatives.  Similarly,  in  our  1988  report  on 
the  first  F-16  multiyear  procurement,  we  concluded  that  savings  were 
likely  achieved  from  EOQ  buys  in  the  order  of  magnitude  expected,  but  we 
were  unable  to  make  definitive  conclusions  about  total  savings  achieved.21 


Undergirding  attempts  to  track  and  assess  multiyear  performance  is  the 
quality  and  sufficiency  of  data.  As  discussed  earlier  in  this  report,  we 
found  that  DOD  at  all  organizational  levels  does  not  keep  adequate  records 
on  multiyear  programs  to  document  stakeholder  reviews  and  the  empirical 
evidence  used  to  justify  multiyear  contracts.  OSD  Comptroller  officials 
told  us  that,  while  they  are  the  final  authority  on  approving  multiyear 
candidates  and  the  ultimate  owner  of  the  review  process,  they  do  not  track 
multiyear  packages  through  the  approval  process  and  after  the  final 
decision  has  been  made  to  submit  it  to  the  Congress.  DOD  has  no 
comprehensive,  central  information  system  that  records  the  status  of 
multiyear  candidates  in-process,  candidates  that  have  been  approved  or 
disapproved,  detailed  information  on  how  multiyear  criteria  were  applied, 
or  information  on  specific  criteria  contained  in  the  final  justification 
package. 

DOD  is  not  required  to,  nor  does  it  maintain  a  central  database  of 
historical  or  ongoing  multiyear  contracts.  In  response  to  our  inquiries,  the 
OSD  office  ultimately  responsible  for  the  multiyear  procurement  review 
process  was  unable  to  provide  us  the  justification  packages  for  over  half 
of  the  programs  approved  by  Congress  since  1992,  including  very  recent 
submissions.  To  obtain  more  complete  and  accurate  data  that  could  be 
used  to  track  performance  and  conduct  trend  analyses,  information  must 
be  compiled  from  many  different  sources,  including  budgets,  program 
office  files,  contractor  studies,  and  contracting  databases.  For  example, 
our  efforts  to  identify  and  track  multiyear  contract  information  contained 
in  two  major  federal  databases22  were  only  partially  successful.  Some 
valuable  multiyear  contract  information  was  readily  available  through 
these  sources.  However,  the  type  of  data  stored  and  storage  format — as 


Inadequate  Records 
Maintained  on  Current  and 
Historical  Multiyear 
Programs 


21  GAO,  Procurement.:  An  Assessment  of  the  Air  Force’s  F-l  6  Aircraft  Multiyear  Contract, 
GAO/N SIAD-86-39  (Washington,  D.C.:  Feb.  20,  1986). 

22  The  two  databases  were  the  DOD  DD360  Individual  Contract  Action  Report,  and  the 
Federal  Procurement  Data  System-Next  Generation  (FPDS-NG). 
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well  as  issues  pertaining  to  reliability,  consistency,  and  comparability — 
limited  their  current  usefulness  in  tracking  and  evaluating  multiyear 
contracts.  However,  with  some  improvements  particularly  with  reliability, 
these  databases  could  support  future  studies. 


Conclusions 


The  statutory  criteria  for  approving  the  multiyear  procurement  of  major 
DOD  weapon  systems  clearly  establish  requirements  to  limit  multiyear 
authorization  to  stable,  low  risk  programs  with  realistic  cost  estimates  and 
reasonable  expectation  for  savings.  To  move  forward  otherwise  is  to 
accept  significant  risks  with  little  chance  of  reward.  However,  DOD  does 
not  have  an  adequate  process  with  controls  in  place  to  ensure  multiyear 
candidates  meet  all  the  criteria  and  are  supported  by  sufficient  empirical 
evidence.  Inconsistent  application  of  criteria,  questionable  cost  and 
savings  estimates,  and  inadequate  documentation  increase  potential  for 
approving  inappropriate,  unstable  multiyear  programs  and  incurring 
costly,  poor  outcomes  when  plans  go  awry  and  conditions  change. 
Improving  guidance,  ensuring  decisions  are  informed  by  knowledge,  and 
maintaining  better  records  are  critical  needs,  as  well  as  important  tools  for 
retaining  corporate  memory  given  frequent  turnover  of  personnel  at  all 
levels  of  the  justification  review  process.  It  is  not  possible  to  calculate 
accurately  the  cost  to  taxpayers  that  has  resulted  from  these  conditions, 
but  the  lack  of  a  disciplined  and  rigorous  process  that  demands  knowledge 
about  stability  and  costs  provides  potential  for  significant  waste  of 
taxpayer  dollars. 

Furthermore,  DOD  does  not  track  and  evaluate  actual  performance  on 
multiyear  contracts  for  major  DOD  weapon  systems.  Once  a  contract  is 
awarded  for  a  multiyear  program,  little  effort  is  made  to  collect  data  and 
assess  actual  results  to  compare  performance  against  original 
expectations  and  to  validate  savings  and  other  benefits  achieved. 

Assessing  results  could  provide  valuable  insights  and  lessons  learned  on 
prior  experience  and  identify  opportunities  to  improve  future  multiyear 
procurements.  Not  having  a  clear  picture  of  actual  performance  further 
emphasizes  the  criticality  of  getting  it  right  up  front  by  ensuring  only 
appropriate  programs  go  to  Congress  for  approval. 

Therefore,  despite  a  long  history  and  substantial  funding  for  major  DOD 
weapons  system  multiyear  contracts,  DOD  does  not  know  whether  it  has 
gotten  a  reasonable  return  on  its  investments  for  the  extra  risks  incurred. 
Some  concerns  noted  in  this  report,  such  as  the  practice  of  understating 
costs  and  overselling  benefits,  apply  also  to  annual  contracts,  but  the 
standards  should  be  higher  for  multiyear  contracts  because  of  the  larger 
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up-front  investments  required  and  the  government’s  exposure  to  risk 
should  the  program  fail  or  be  substantially  changed.  Strengthening  both 
the  front  end  of  the  process — identifying  and  justifying  good  candidates — 
and  the  back  end — assessing  results  and  gleaning  lessons  learned  from 
completed  contracts — can  help  ensure  costs  and  risks  are  adequately 
balanced  for  new  multiyear  programs  and  improve  future  outcomes. 
Underpinning  both  ends,  it  is  important  to  capture  and  make  available 
essential  data  on  multiyear  decisions  and  subsequent  performance  that 
can  be  readily  accessed  by  stakeholders  and  prospective  users  of 
multiyear  procurement  authority. 


Recommendations 


To  improve  the  outcomes  of  the  multiyear  justification  reviews  of  major 
DOD  weapon  systems,  the  Secretary  of  Defense  should  direct  appropriate 
offices  within  the  Under  Secretary  of  Defense  (USD)  (Comptroller)  and 
USD  (Acquisition,  Technology  &  Logistics)  to:  (1)  improve  and  expand 
guidance  provided  to  military  services  to  better  define  multiyear  decision 
criteria  for  major  DOD  weapon  systems  and  to  facilitate  more  consistent, 
objective,  and  knowledge-based  evaluations  of  these  multiyear  candidates 
within  DOD;  (2)  establish  a  process  for  third  party  validation  of  the  costs 
and  savings  data  submitted  for  such  candidate  programs;  and  (3) 
implement  a  central  database  for  maintaining  historical  records  and  for 
effectively  monitoring  and  tracking  major  DOD  weapon  system  multiyear 
procurements,  to  include  documenting  the  specific  decisions  made  by 
stakeholders  and  their  rationales  for  decisions. 

To  provide  lessons  learned  for  informing  and  improving  future  major  DOD 
weapon  system  multiyear  candidate  programs  and  to  ensure  DOD  is 
earning  a  sufficient  return  on  its  investments  in  multiyear  contracts  for 
major  DOD  weapon  systems,  the  Secretary  of  Defense  should  direct  that 
the  responsible  military  service,  in  conjunction  with  appropriate  elements 
within  OSD,  conduct  after-action  assessments  at  the  conclusion  of  all 
multiyear  contracts  used  to  procure  major  DOD  weapon  systems  to 
determine  their  effectiveness  in  achieving  predicted  benefits  while 
managing  associated  risks.  These  assessments  should  identify  major 
deviations,  if  any,  between  the  unit  costs  predicted  in  the  multiyear 
justification  package  and  the  unit  costs  actually  incurred.  The  assessments 
should  also  substantiate — to  the  extent  practicable — savings  achieved  and 
identify  reasons  and  causes  contributing  to  overall  performance  results 
and  attemp  to  isolate  those  issues  peculiar  to  the  multiyear  contract  from 
those  that  would  likely  have  also  affected  annual  contracts  if  a  multiyear 
strategy  had  not  been  employed.  Internally,  DOD  should  use  the  results  of 
these  assessments  to  provide  lessons  learned  to  both  industry  and  the 
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government  that  can  help  inform  and  lead  to  better  and  more  supportable 
decisions  on  future  multiyear  candidate  programs. 


DOD  provided  us  with  written  comments  on  a  draft  of  this  report.  The 
comments  appear  in  appendix  II.  DOD  also  separately  provided  technical 
comments  which  we  reviewed  and  incorporated  as  appropriate.  In  written 
comments,  DOD  concurred  with  our  two  recommendations  to  improve 
guidance  and  to  implement  a  central  database  for  maintaining  records  and 
tracking  multiyear  programs.  DOD  partially  concurred  with  the  other  two 
recommendations. 

DOD  partially  concurred  with  our  recommendation  to  establish  a  process 
for  third  party  validation  of  the  costs  and  savings  data  submitted  for 
candidate  programs.  In  its  comments,  DOD  stated  that  independent  third 
party  validations  of  cost  and  savings  are  done  on  selected  programs  and, 
in  developing  new  guidance  in  response  to  our  first  recommendation, 
would  consider  whether  the  benefits  of  requiring  validation  on  all 
programs  warrant  the  delays  and  costs  of  validation.  Our  review  of  five 
new  proposals  and  six  approved  multiyear  contracts  found  only  one  such 
instance,  and  our  discussions  with  service  and  OSD  officials  show  that 
third  party  validations  are  rare.  We  believe  that  independent  third  party 
validations  would  result  in  more  accurate  and  comprehensive  cost  and 
savings  information  critical  to  congressional  and  DOD  decision  making  on 
multiyear  candidates.  Our  review  identified  inconsistent  practices  in 
preparing  and  reviewing  multiyear  proposals  and  varying  degrees  of 
quality  and  completeness  in  the  initial  cost  and  savings  estimates  made  by 
the  weapon  system  program  offices.  An  independent  third  party  check 
would  help  ensure  that  appropriate  multiyear  candidates  are  submitted  to 
the  Congress  for  approval. 

DOD  also  partially  concurred  with  our  recommendation  that  after-action 
assessments  be  conducted  to  provide  lessons  learned  for  informing  and 
improving  future  multiyear  candidate  programs.  In  its  comments,  DOD 
agreed  that  after-action  reports  may  be  of  value  for  certain  multiyear 
programs,  but  questioned  the  value  for  all  programs  because  (1)  the 
extensive  time  before  the  assessment  results  are  known  and  can  be 
applied;  and  (2)  the  difficulty  in  determining  actual  savings.  The  intent  of 
the  recommendation  is  to  learn  lessons  that  can  be  applied  to  strengthen 
future  multiyear  proposals  and  improve  their  prospects  for  success.  This 
in  formulation  is  not  time-bounded  as  DOD  has  contracted  for  studies  that 
drew  useful  lessons  from  programs  many  years  earlier.  Collecting  and 
distributing  data  on  lessons  learned  would  provide  a  continuing  database 
of  knowledge  for  future  programs.  Furthermore,  while  we  recognize 
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difficulties  and  constraints  in  calculating  actual  savings,  this  does  not 
preclude  the  department  from  making  good  faith  efforts  that  can  provide 
valuable,  albeit  imperfect  information.  Also,  after-action  assessments 
include  more  than  savings  calculations.  As  pointed  out  in  the  report,  one 
program  assessed  itself  and  identified  important  contractual  features  and 
other  factors  that  it  used  to  improve  subsequent  multiyear  proposals.  We 
note  that  the  practice  of  doing  after-action  reports  is  widespread  in  the 
department  and  used  for  many  different  kinds  of  activities,  including 
military  contingency  operations  and  logistics  functions,  and  that  these 
efforts  provide  planners  and  decision  makers  with  critical  lessons  learned 
for  applying  to  and  improving  future  actions. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  and  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy;  and  the  Director  of  the 
Office  of  Management  and  Budget.  We  will  provide  copies  to  others  on 
request.  This  report  will  also  be  available  at  no  charge  on  GAO’s  Web  site 
at  http://www.gao.gov.  Key  contributors  to  this  report  were  Bruce 
Fairbairn,  Assistant  Director;  Noah  Bleicher;  Matthew  Drerup;  Mary  Jo 
Lewnard;  Rae  Ann  Sapp;  and  Bob  Swierczek. 

If  you  have  any  questions  about  this  report  or  need  additional  information, 
please  call  me  at  (202)  512-4841  (sullivanm@gao.gov).  Contact  points  for 
the  offices  of  Congressional  Relations  and  Public  Affairs  are  located  on 
the  last  page  of  this  report. 


Michael  J.  Sullivan 
Director,  Acquisition 
and  Sourcing  Management 
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Our  review  was  limited  to  major  DOD  weapon  systems  that  have  received 
congressional  approval  in  annual  defense  authorization  and/or 
appropriations  acts  to  award  a  multiyear  contract  and  meet  the  statutory 
requirements  identified  in  10  U.S.C.  §  2306b.  Work  was  performed  at  the 
Offices  of  the  Secretary  of  Defense  and  the  three  military  service 
headquarters  (Navy,  Army,  and  Air  Force),  in  Washington,  D.C.;  Naval  Air 
Systems  Command,  in  Patuxent  River,  Maryland;  Aviation  Missile 
Command,  in  Huntsville,  Alabama;  Aeronautical  Systems  Center,  at 
Wright-Patterson  Air  Force  Base,  Ohio;  Institute  of  Defense  Analysis 
(IDA),  Alexandria,  Virginia;  and  RAND  National  Defense  Research 
Institute,  Arlington,  Virginia. 

To  evaluate  DOD’s  multiyear  review  process,  we  compiled  a  list  of  the  all 
the  candidate  programs  approved  by  Congress  to  use  a  multiyear  contract 
by  examining  DOD  authorization  and  appropriations  acts  going  back  to 
fiscal  year  1982  when  the  statutory  language  in  10  U.S.C  §  2306b  was  first 
enacted.  We  then  reviewed  DOD  multiyear  justification  packages 
submitted  to  Congress  in  recent  defense  budgets,  identified  statutory 
criteria  authorizing  the  use  of  multiyear  procurement,  and  considered 
regulatory  policies  and  procedures  used  within  the  Services  and  at  OSD  to 
prepare  and  evaluate  multiyear  justification  packages.  We  discussed  with 
officials  at  acquisition  program  offices  and  at  higher  command  review 
levels  how  they  interpreted  and  applied  the  statutory  criteria  and  guidance 
to  evaluate  the  appropriateness  and  feasibility  of  multiyear  candidates.  We 
reviewed  DOD  and  congressional  actions  on  recent  multiyear  candidates 
and  examined  specifically  two  major  programs  recently  approved  for 
multiyear  contracting — the  F-22  Raptor  and  V-22  Osprey — to  illustrate  how 
this  process  works  and  address  some  questions  raised  about  the 
appropriateness  of  these  candidates  and  data  used  in  the  justification 
packages.  For  these  systems,  we  extensively  drew  upon  GAO’s  work  in 
prior  and  ongoing  engagements. 

We  conducted  limited  case  studies  for  selected  multiyear  aircraft 
contracts  to  assess  outcomes  and  the  internal  and  external  events 
affecting  performance.  Because  our  sample  of  DOD  aircraft  multiyear 
contracts  was  not  randomly  selected,  we  cannot  project  our  findings  to 
other  programs.  These  case  studies  included  three  major  DOD  weapon 
systems — C-17A  Globemaster,  F/A-18E/F  Super  Hornet,  and  the  Apache 
Longbow  Helicopter.  These  aircraft  programs  have  fully  executed  at  least 
one  multiyear  contract  in  the  recent  past  and  also  have  ongoing  follow-on 
multiyear  contracts.  We  reviewed  the  multiyear  proposal  packages 
submitted  to  Congress,  annual  budget  information,  Selected  Acquisition 
Reports,  contract  file  documentation,  and  information  in  DOD  contract 
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databases  for  multiyear  contracts  awarded  by  these  programs.  We 
calculated  unit  cost  changes  and  identified  key  programmatic  and 
environmental  changes  impacting  the  execution  of  the  multiyear  programs 
and  compared  these  to  the  original  projections  in  their  multiyear 
justification  packages.  To  identify  the  characteristics  of  multiyear 
contracts  and  how  they  affect  the  costs,  risks,  and  savings  for  selected 
multiyear  aircraft  contracts,  we  reviewed  contract  file  documentation  and 
information  on  the  DOD  DD  350  Individual  Contract  Action  Report  (ICAR) 
database  and  the  Federal  Procurement  Data  System-Next  Generation 
(FPDS-NG).  As  part  of  the  case  study  approach,  we  reviewed  the  types  of 
contracts,  contract  clauses,  and  other  contract  modifications  to  determine 
how  they  affected  the  unit  costs  under  a  multiyear  contract.  Further,  we 
discussed  the  multiyear  contracts  included  in  our  limited  case  study 
approach  and  the  F-22A  program’s  August  2007  multiyear  contract  award 
with  program  officials  to  help  us  assess  the  effects  that  specific  contract 
provisions  have  on  unit  costs  during  contract  execution. 

To  research  the  legislative  history  underlying  the  adoption  and  subsequent 
repeal  of  the  10-percent  savings  requirement  and  the  current  requirement 
that  defense  multiyear  procurement  contracts  achieve  “substantial 
savings,”  we  reviewed  the  evolution  of  multiyear  savings  criteria  in  early 
defense  initiatives,  the  fiscal  year  1982  codification,  subsequent 
amendments,  and  the  savings  criteria  contained  in  annual  authorization 
and  appropriations  acts  subsequent  to  the  granting  of  multiyear  authority. 
To  make  comparisons  in  average  and  median  levels  of  estimated  savings 
on  multiyear  candidate  programs  as  the  criteria  for  awarding  a  multiyear 
contract  changed,  we  reviewed  savings  estimates  in  the  budget 
justification  packages  for  multiyear  candidates  submitted  to  Congress 
since  1982  and  past  GAO  reviews  of  multiyear  candidates  conducted  in  the 
1980s.  We  were  able  to  obtain  savings  estimates  for  approximately  94  of 
the  141  approvals  granted  by  Congress  to  award  a  multiyear  contract. 

To  determine  the  extent  to  which  DOD  tracks  performance  and  validates 
savings  and  other  benefits  actually  achieved  by  multiyear  contracts,  we 
evaluated  the  kind  and  extent  of  cost  data  and  program  information 
maintained  at  the  services  and  OSD,  and  how  they  use  these  data  to 
determine  whether  predicted  savings  and  other  benefits  were  actually 
achieved  by  multiyear  contracting.  We  also  reviewed  DOD  policies  and 
guidance  for  estimating  and  validating  multiyear  savings,  and  discussed 
with  DOD  officials  practices  used  to  estimate  and  monitor  multiyear 
savings.  We  discussed  with  DOD  officials  record-keeping  and  management 
oversight  requirements  and  reasons  why  they  do  not  have  a  formal  process 
for  assessing  multiyear  results.  We  reviewed  two  recent  major  studies 
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done  by  defense  research  firms  that  summarized  estimated  savings  on 
historical  programs  and  performed  case  studies  on  selected  programs  to 
identify  key  events  and  practices  that  affected  ultimate  performance.  We 
discussed  with  IDA  and  RAND  officials  their  cost  estimating  techniques 
and  assumptions  used  in  their  F-22A  multiyear  studies  that  supported  the 
planned  multiyear  contract.  We  reviewed  and  summarized  their  attempts 
to  validate  savings  and  other  benefits  from  prior  multiyear  programs.  We 
also  reviewed  prior  GAO  work  on  selected  weapon  systems  and  the  results 
of  our  work  during  the  1980s  when  Congress  regularly  asked  us  to  review 
DOD’s  multiyear  candidates. 

We  conducted  this  performance  audit  from  June  2007  to  February  2008  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON  DC  20301-3000 


ACQUISITION 
TECHNOLOGY 
AND  LOGISTICS 

FEB  0  1  2008 

Mr.  Michael  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report, 
GAO-08-298,  “DEFENSE  ACQUISITIONS:  DoD’s  Practices  and  Processes  for 
Multiyear  Procurement  Should  Be  Improved,”  dated  December  21,  2007,  (GAO  Code 
120649). 

The  Department  acknowledges  receipt  of  the  draft  report.  DoD’s  written  comments 
are  enclosed. 

Sincerely, 

,/p  . 

/  /  / 

A  /dA&td 

.  ^j\  Shay  D.  Assad, 

1  Director,  Defense  Procurement 
and  Acquisition' Policy 

Enclosure: 

As  stated 


/HiT0 
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GAO  Draft  Report  Dated  December  21,  2007 
GAO-08-298  (GAO  CODE  120649) 

“DEFENSE  ACQUISITIONS:  DOD’S  PRACTICES  AND  PROCESSES  FOR 
MULTIYEAR  PROCUREMENT  SHOULD  BE  IMPROVED” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 

POD  GENERAL  COMMENTS.  The  department  is  strongly  committed  to  multiyear 
(MY)  procurements  which  (1)  offer  substantial  savings  through  improved  economies  and 
efficiencies  in  production  processes,  and  better  utilization  of  industrial  facilities;  (2) 
enhance  the  attractiveness  of  and  competition  for  Government  requirements  at  the 
subcontracting  level;  and  (3)  reduce  the  administrative  burden  in  the  placement  and 
administration  of  contracts.  A  key  factor  in  the  successful  use  of  MY  procurements  is  the 
intelligent  selection  of  the  programs. 

In  Appendix  I:  Scope  &  Methodology,  GAO  indicated  that  their  review  was  limited  to 
major  DOD  weapon  systems.  Accordingly,  the  department  suggests  that  the 
recommendations  only  be  applied  to  this  category  of  MY  procurements,  and  not  to  other 
categories,  e.g.,  procurements  of  energy  commodities  and  energy-related  services. 

The  description  of  the  “stability  of  funding”  criteria  in  Table  1  of  the  report  should  be 
revised  to  more  accurately  reflect  the  intent  of  statute  (10  USC  2306b(a)(3)).  The  phrase 
“the  head  of  the  agency  will  request  funding”  rather  than  the  phrase  “sufficient  funding 
will  be  provided  by  DOD”  better  reflects  the  statute.  Likewise,  the  description  of  the 
“stable  design”  criteria  in  the  same  table  should  be  revised  by  substituting  the  phrase 
“technical  risks  that  are  not  excessive”  for  the  phrase  “low  technical  risk”  (10  USC 
2306b(a)(4)). 

The  department  will  also  comply  with  the  recent  amendments  to  the  multiyear  statute,  10 
U.S.C.  2306b,  made  by  section  81 1  of  the  National  Defense  Authorization  Act  for  Fiscal 
Year,  2008. 

RECOMMENDATION  1:  The  GAO  recommended  that  the  Secretary  of  Defense  should 
direct  appropriate  offices  within  the  Under  Secretary  of  Defense  (USD)(Comptroller)  and 
USD  (Acquisition,  Technology  and  Logistics)  to  improve  and  expand  guidance  provided 
to  military  services  to  better  define  MY  decision  criteria  and  to  facilitate  more  consistent, 
objective,  and  knowledge-based  evaluations  of  MY  candidates  within  the  department. 

(p. 2 8/GAO  Draft  Report) 

DOD  RESPONSE:  Concur.  To  facilitate  evaluations  of  MY  candidates  at  the  time  they 
are  proposed  and  prior  to  a  request  for  MY  contract  authority,  guidance  is  needed  to  better 
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define  the  Office  of  the  Secretary  of  Defense’s  review  process.  The  guidance  will  result  in 
an  enhanced  deliberative  process  for  the  evaluation  of  multiyear  candidates,  and  improved 
historical  records  that  document  the  reviews  and  the  supporting  rationales  for  the 
decisions.  The  department  plans  to  issue  this  guidance  within  one  year  after  the  final  GAO 
report  is  published. 

RECOMMENDATION  2:  The  GAO  recommended  that  the  Secretary  of  Defense  should 
direct  appropriate  offices  within  the  USD  (Comptroller)  and  USD  (Acquisition, 
Technology  and  Logistics)  to  establish  a  process  for  third  party  validation  of  the  costs  and 
savings  data  submitted  for  candidate  programs,  (p.  28/GAO  Draft  Report) 

POD  RESPONSE.  Partially  concur.  Costs  and  savings  are  reviewed  twice;  once  when 
MY  authority  is  proposed  and  again  at  the  conclusion  of  the  contract  negotiations  when  it 
is  determined  that  the  costs  and  savings  meet  the  requirements  of  the  statute. 

OSD’s  Cost  Analysis  Improvement  Group  or  one  of  several  Federally  Funded 
Research  and  Development  Centers  (FFRDC)  perform  an  independent  cost  assessment  on 
selected  programs.  Also,  the  procedures  for  negotiating  a  major  contract  include 
consideration  of  independent  cost  audits  and  assessments  from  the  Defense  Contract 
Management  Agency  and  the  Defense  Contract  Audit  Agency. 

When  developing  OSD’s  guidance  described  in  response  to  recommendation  1,  the 
Department  will  consider  whether  the  benefits  of  requiring  third  party  validation  of 
preliminary  estimates  of  costs  and  savings  data  for  all  programs  warrant  the  delays  and  (he 
expense  of  validation,  since  a  final  determination  of  statutory  compliance  must  be  made  as 
part  of  the  contract  negotiation  and  award  process. 

RECOMMENDATION  3 :  The  GAO  recommended  that  the  Secretary  of  Defense  should 
direct  appropriate  offices  within  the  USD  (Comptroller)  and  USD  (Acquisition, 
Technology  and  Logistics)  to  implement  a  central  database  for  maintaining  historical 
records  and  for  effective  monitoring  and  tracking  multiyear  programs,  to  include 
documenting  the  specific  decisions  made  by  stakeholders  and  their  rationales  for  decisions, 
(p.  28/GAO  Draft  Report) 

POD  RESPONSE.  Concur.  DOD  will  implement  this  action  concurrent  with 
implementation  of  the  guidance  described  in  DOD  response  to  recommendation  1 . 

RECOMMENDATION  4:  The  GAO  recommended  that  the  Secretary  of  Defense  direct 
that  the  responsible  military  service,  in  conjunction  with  appropriate  elements  within  OSD, 
conduct  an  after  action  assessment  at  the  conclusion  of  each  multiyear  contract  to 
determine  its  effectiveness  in  achieving  predicted  benefits  while  managing  associated 
risks,  (p.  28/GAO  Draft  Report) 
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POD  RESPONSE.  Partially  concur.  The  department  concurs  that  an  after  action 
assessment  may  be  of  value  for  certain  programs.  However,  in  general,  it  is  not  known 
whether  the  value  in  performing  an  assessment  (e.g.,  lessons  learned)  on  all  MY 
programs  outweighs  (1)  the  extensive  time  before  the  assessment  results  are  known  and 
can  be  applied  to  similar  programs;  and  (2)  the  difficulty  in  determining  actual  savings. 
First,  the  assessment  can  not  begin  until  after  performance  is  complete  and  data  on  actual 
costs  is  available,  e.g.,  7-8  years  after  contract  award.  It  is  not  apparent  how  valuable  the 
results  would  be  to  the  department  after  such  a  long  period  has  transpired.  Second,  as 
indicated  in  the  report,  savings  are  difficult  to  determine  due  to  too  many  factors  that  may 
affect  costs  during  the  life  of  a  MY  contract,  e.g.,  changes  in  quantity,  economic 
conditions,  or  configuration  (e.g.,  to  address  safety,  reliability,  or  performance  problems); 
the  impact  of  cost  reduction  initiatives,  subsequent  changes  in  statutory  and  regulatory 
rules  that  may  impact  costs,  etc.  As  recognized  in  the  report,  calculating  the  cost  of  a 
MY  contract  if  these  unanticipated  changes  had  not  occurred  would  be  especially 
challenging  and  difficult  to  validate.  Likewise,  it  is  difficult  to  determine  how  an 
alternative  series  of  annual  contracts  would  have  fared  under  these  conditions. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday,  GAO  posts 
newly  released  reports,  testimony,  and  correspondence  on  its  Web  site.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products  every  afternoon,  go 
to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each. 

A  check  or  money  order  should  be  made  out  to  the  Superintendent  of 
Documents.  GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or 
more  copies  mailed  to  a  single  address  are  discounted  25  percent.  Orders 
should  be  sent  to: 

U.S.  Government  Accountability  Office 

441  G  Street  NW,  Room  LM 

Washington,  DC  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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